Learning Efficient GANs for Image Translation via Differentiable Masks and co-Attention Distillation

Generative Adversarial Networks (GANs) have been widely-used in image translation, but their high computational and storage costs impede the deployment on mobile devices. Prevalent methods for CNN compression cannot be directly applied to GANs due specificity of GAN tasks unstable adversarial training. To solve these, this paper, we introduce a novel method, termed DMAD, by proposing Differentiable Mask co-Attention Distillation. The former searches light-weight generator architecture training-adaptive manner. overcome channel inconsistency when pruning residual connections, an adaptive cross-block group sparsity is further incorporated. latter simultaneously distills informative attention maps from both discriminator pre-trained model searched generator, effectively stabilizing training our model. Experiments show that DMAD can reduce Multiply Accumulate Operations (MACs) CycleGAN 13x Pix2Pix 4x while retaining comparable performance against full Our code available at https://github.com/SJLeo/DMAD.